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Abstact 

The  NASA  Tetrahedral  Unstructured  Software 
System  (TetrUSS)  was  developed  during  the 
1990’ s  to  provide  a  rapid  aerodynamic  analysis 
and  design  capability  to  applied 
aerodynamicists.  The  system  is  comprised  of 
loosely  integrated,  user-friendly  software  that 
enables  the  application  of  advanced  Euler  and 
Navier-Stokes  tetrahedral  finite  volume 
technology  to  complex  aerodynamic  problems. 
TetrUSS  has  matured  well  because  of  the 
generous  feedback  from  many  willing  users 
representing  a  broad  cross-section  of 
background  and  skill  levels.  This  paper  presents 
an  overview  of  the  current  capabilities  of  the 
TetrUSS  system  along  with  some  representative 
results  from  selected  applications. 

1  Introduction 

The  last  decade  of  the  twentieth  century  brought 
about  remarkable  progress  in  practical 
computational  aerodynamic  analysis  and  design 
capability.  This  phenomenon  was  largely  fueled 
by  breakthrough  developments  in  the  science  of 
Computational  Fluid  Dynamics  (CFD), 
geometric  modeling,  computer  architecture,  and 
networking  speeds.  Intense  competitive 
pressures  to  build  more  efficient  or  specialized 
products,  e.g.  aircraft,  automobiles,  buildings, 
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pumps,  heat  exchangers,  etc.,  in  less  time  and  at 
lower  cost  and  risk,  further  drove  demand  for 
more  timely  solutions  to  fluid  dynamic 
problems.  Consequently,  CFD  tools  are  now  a 
critical  component  of  many  design  processes 
that,  in  the  past,  relied  solely  on  empirical  and 
experimental  techniques. 

The  underlying  methodologies  for  CFD 
have  evolved  over  the  past  30  years  through  the 
efforts  of  many  talented  researchers.  The 
majority  of  work  has  centered  on  the  structured- 
grid  approach  for  solving  the  Euler  and  Navier- 
Stokes  flow  equations,  where  discretization  of 
space  is  accomplished  by  hexahedral  volume 
elements.  The  natural  ordering  of  hexahedral 
cells  enables  the  construction  of  very  efficient 
numerical  algorithms  for  solving  the  flow 
equations.  While  amenable  to  algorithmic 
efficiency,  structured- grids  are  inherently 
difficult  to  construct  around  complex 
geometries.  Innovative  remedies  were  devised 
for  decomposing  the  global  grid  into  simpler 
component  grids  that  could  be  generated  more 
readily.  Communication  between  component 
grids  is  achieved  either  by  direct  transfer  across 
interfacing  boundaries  (patched  grids),  or  by 
interpolation  within  overlapping  grid  regions 
(overset  grids).  These  extensions  are  responsible 
for  the  continued  success  of  the  structured- grid 
codes  today,  particularly  for  commercial 
transport  aircraft  companies  where  the 
underlying  configuration  has  changed  very  little 
over  time. 
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For  more  complex  and  unconventional 
geometries,  structured  grid  generation  times 
routinely  extend  into  weeks  or  months.  Because 
of  the  growing  demand  for  more  timely 
application  of  CFD  tools  to  increasingly 
complex  and  unique  problems,  researchers 
began  during  the  mid-  to  late- 1980’ s  to  explore 
the  unstructured- grid  approach  using  triangular 
elements  in  two  dimensions  and  tetrahedral  cells 
in  three-dimensions.  These  elements  had  been 
used  in  the  finite-element  structural  analysis 
community  for  many  years,  and  were  known  to 
be  exceptionally  adaptable  for  discretizing 
complex  shapes.  While  several  unstructured- 
grid  methodologies  have  realized  considerable 
success  in  recent  years,  their  underlying 
algorithms  are  still  relatively  new  and  diverse. 
There  are  still  many  philosophical  differences 
among  researchers  regarding  the  superiority  of 
one  approach  over  another.  A  good  overview  of 
the  technology  is  presented  in  Ref.  [1]. 

Some  of  the  most  enabling  advancements 
toward  rapid  CFD  application  have  occurred  in 
the  area  of  tetrahedral  grid  generation  [2-6], 
With  the  advent  of  the  Advancing  Front  Method 
[2]  and  the  Advancing-Layers  Method  [3], 
‘viscous’  tetrahedral  grids  can  now  be  generated 
on  complex,  full  aircraft  configurations  within 
days. 

With  the  recent  advancements  in 
unstructured  CFD  technology,  a  tremendous 
benefit  is  gained  by  harnessing  the  various 
component  technologies  into  a  user-friendly 
system.  The  NASA  Tetrahedral  Unstructured 
Software  System  ( TetrUSS )  was  developed  at 
NASA  Langley  Research  Center  during  the 
1990’s  to  address  this  need.  TetrUSS  is  a  loosely 
integrated  unstructured-grid  based  CFD  system 
that  provides  ready  access  to  rapid,  higher-order 
analysis  and  design  capability  for  the  applied 
aerodynamicist.  From  a  developer’s  perspective, 
the  maturation  of  the  underlying  methodology  is 
greatly  accelerated  by  encouraging  feedback 
from  user  applications  of  the  system  to  focused 
problems.  While  specific  codes  are  currently 
used  within  the  TetrUSS  system,  it  is 
sufficiently  modular  such  that  other  component 


codes  can  be  easily  interchanged  if  desired.  The 
primary  value  of  the  present  work  stems  from 
the  engineering  capability  derived  by  the 
integration  of  state-of-the-art  CFD  technologies. 

2  Overview  of  Current  Capability 

The  TetrUSS  system,  illustrated  in  Fig.  1, 
consists  of  component  software  for  setting  up 
geometric  surface  definitions,  generating 
discrete  computational  grids,  computing  Euler 
and/or  Navier-Stokes  flow  solutions,  imposing 
design  or  aeroelastic  shape  changes,  and 
extracting  meaningful  information  for  analysis 
of  the  results.  The  primary  component  codes 
within  the  TetrUSS  system  are  denoted  in  Fig.  1 
as  GridTool,  VGRIDns,  USM3Dns,  and 
VIGPLOT.  However,  as  evidenced  by  several 
other  codes  included  within  parenthesis,  the 
system  is  sufficiently  flexible  that  other 
researchers  have  also  benefited  by  replacing 
various  components  with  their  own  codes.  The 
capabilities  of  the  current  system  are  described 
in  the  following  subsections. 

2.1  Grid  Generation 

2.1.1  GridTool 

GridTool  [7]  is  an  interactive  program  for 
grid/geometry  applications  which  bridges  the 
gap  between  Computer  Aided  Design  (CAD) 
representations  and  grid  generation  systems.  It 
supports  many  useful  functions  for  modifying  or 
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Fig.  1  The  Tetrahedral  Unstructured  Software  System 
(TetrUSS). 
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repairing  surface  definitions,  as  well  as  for 
preparing  an  input  file  for  VGRIDns.  While 
several  file  formats  are  readable  by  the  software, 
the  Initial  Graphics  Exchange  Specifications 
(IGES)  format  [8]  is  most  widely  used  as  an 
interface  between  CAD  and  GridTool. 

2.1.2  VGRIDns/POSTGRID 
VGRIDns  is  a  computer  program  for  automatic 
generation  of  tetrahedral  unstructured  grids 
suitable  for  computing  Euler  and  Navier-Stokes 
flow  solutions.  The  methodology  is  based  on  the 
Advancing-Front  method  (AFM)  [2]  and  the 
Advancing-Layers  method  (ALM)  [3].  Both 
techniques  are  based  on  marching  processes  in 
which  tetrahedral  cells  grow  on  an  initial  front 
(triangular  boundary  mesh)  and  gradually 
accumulate  in  the  field  around  the  subject 
geometry.  Unlike  the  conventional  AFM,  which 
introduces  cells  in  the  field  in  a  totally 
unstructured  manner,  the  ALM  generates  layers 
of  thin  tetrahedral  cells  in  a  more  orderly 
fashion  (one  layer  at  a  time)  while  maintaining 
the  flexibility  of  the  AFM.  Once  the  advancing 
front  process  is  completed  in  VGRIDns,  an 
additional  postprocessing  step  is  required  using 
POSTGRID  to  close  any  open  pockets  and  to 
improve  grid  quality. 

The  input  data  file  to  VGRIDns  is  set  up  in 
GridTool.  A  number  of  contiguous  surface 
patches  are  constructed  over  the  underlying 
IGES  surface  definition.  The  grid  characteristics 
such  as  spacing  and  stretching  parameters,  rate 
of  growth,  etc.  are  also  prescribed  by  the  user 
with  GridTool  to  complete  the  input  data  file. 
The  process  of  geometry  and  grid  parameter 
preparation  with  GridTool  constitutes  50-90% 
of  the  total  grid-generation  time  depending  on 
the  complexity  of  the  geometry  and  the 
"cleanliness"  of  the  surface  definition. 

Cartesian  background  grid  -  Information 
regarding  the  unstructured  grid  point 
distribution  is  provided  to  the  AFM  and  ALM 
marching  processes  by  a  "transparent"  Cartesian 
background  grid  overlaying  the  entire  domain 
[9].  Included  in  the  background  grid  are  a 
number  of  "point"  and  "line"  sources  prescribed 


by  the  user.  The  sources  contain  information 
about  the  characteristics  of  the  final 
unstructured  grid  to  be  generated.  The 
information  is  first  distributed  smoothly  from 
the  sources  onto  the  background  grid  nodes  by 
solving  a  Poisson  equation.  The  problem  is 
similar  to  that  of  the  heat  transfer  in  a 
conducting  medium.  The  smoothed  parameters 
are  then  interpolated  from  the  background  grid 
(and  sources)  to  the  unstructured  grid  points 
being  generated  on  the  surface  and  in  the  field 
during  the  marching  processes. 

Thin-layer  Navier-Stokes  grid  generation  - 
The  thin-layered  tetrahedral  cells  required  for 
Navier-Stokes  solutions  are  generated  by  the 
ALM  method  [3].  Points  are  distributed  along 
predetermined  direction  vectors  according  to  a 
user-prescribed  stretching  function.  Cell-to-node 
connectivity  is  dictated  by  a  predetermined 
connectivity  pattern.  The  layers  grow  into  the 
field  until  some  limiting  criteria,  based  on  the 
background  grid  information  and/or  proximity 
of  the  approaching  fronts,  prevent  the  layers 
from  further  advancement.  At  this  point,  the 
process  automatically  switches  from  the  ALM 
to  the  AFM  mode  to  generate  regular  grid 
outside  the  boundary  layer.  With  a  common 
background  grid  controlling  both  methods,  the 
transition  from  thin  layers  to  the  regular  grid 
becomes  gradual  and  continuous.  In  addition, 
the  fact  that  the  number  of  layers  varies  from 
one  location  to  another  adds  to  the  flexibility  of 
the  method  and  the  smoothness  of  the  generated 
grids. 

Multidirectional  anisotropic  cell  stretching 
A  salient  feature  of  VGRIDns  is  its  ability  to 
generate  multi-directional,  anisotropic  stretched 
grids  in  which  the  surface  triangles  and 
tetrahedra  in  the  field  are  elongated  in  user- 
prescribed  directions  [10],  With  this  capability, 
fewer  points  are  distributed  in  the  directions  of 
reduced  flow  gradient  without  loss  of  grid 
resolution  in  other  essential  directions.  Grid 
stretching  is  achieved  by  prescribing  a  stretching 
direction  and  two  spacing  factors  (along  and 
normal  to  the  stretching  direction)  for  each 
background  grid  source.  Anisotropic  stretching 
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is  a  powerful  tool  for  generating  efficient 
unstructured  grids.  Application  of  this  option  to 
a  typical  aircraft  configuration  results  in 
reduction  of  grid  size  at  least  by  a  factor  of 
three. 

2.1.3  Grid  Movement 

Grid  movement  capability  is  utilized  for 
problems  such  as  time-dependent  flows 
involving  moving  bodies,  aeroelastic  analysis  of 
aircraft  undergoing  surface  deflections, 
interactive  CFD/design  analysis  resulting  in 
geometry  perturbation,  etc.  An  unstructured 
grid  movement  method  has  been  developed  in 
this  work  to  perturb  tetrahedral  volume  grids 
according  to  the  movement  or  deformation  of 
the  subject  geometries  [3].  The  method  is  based 
on  a  "spring"  analogy  for  moving  inviscid  grids, 
and  a  combination  of  spring  and  "surface  vector 
readjustment"  methods  for  moving  thin-layer 
Navier-Stokes  grids.  The  scheme  is  very  robust 
for  moving  thin  layers  of  viscous  grids.  As  long 
as  the  fidelity  of  the  surface  definition  is 
preserved  after  perturbation,  it  always  produces 
a  valid  grid. 

2.2  Flow  Solver 

2.2.1  USM3Dns  Salient  Features 
USM3Dns  [11,12]  is  a  tetrahedral  cell-centered, 
finite  volume  Euler  and  Navier-Stokes  (N-S) 
flow  solver.  Inviscid  flux  quantities  are 
computed  across  each  cell  face  using  Roe’s  [13] 
flux-difference  splitting  (FDS).  Spatial 
discretization  is  accomplished  by  a  novel 
reconstruction  process  [14],  which  is  based  on 
an  analytical  formulation  for  computing  solution 
gradients  within  tetrahedral  cells.  The  solution 
is  advanced  to  a  steady  state  condition  by  an 
implicit  backward-Euler  time- stepping  scheme 

[15] ,  Flow  turbulence  effects  are  modeled  by  the 
Spalart-Allmaras  (S-A)  one-equation  model 

[16] ,  which  is  coupled  with  a  wall  function  to 
reduce  the  number  of  cells  in  the  sublayer  region 
of  the  boundary  layer. 

USM3Dns  runs  with  multitasking  on  Cray 
vector  processors,  and  more  recently  on 
massively  parallel  processors.  Memory  is 


allocated  dynamically.  The  code  requires  175 
eight-bit  words  per  tetrahedra,  and  runs  with 
individual  processor  times  of  34  jisec/cell/cycle 
on  a  Cray-C90  and  230  jasec/cell/cycle  on  a 
single  CPU  of  an  Origin  2000.  Work  is 
underway  to  port  the  parallel  code  to  personal 
computer  (PC)  clusters. 

2.2.2  Boundary  conditions 
The  USM3Dns  code  is  designed  for  the  easy 
addition/modification  of  boundary  conditions 
(B.C.).  It  supports  the  standard  B.C.’s  of  flow 
tangency  or  no-slip  on  solid  surfaces, 
characteristic  inflow/outflow  for  subsonic 
boundaries,  and  freestream  inflow  and 
extrapolation  outflow  for  supersonic  flow.  Some 
additional  special  boundary  conditions  are 
available  as  well. 

Wall  Function  -  The  S-A  model  has  been 
coupled  with  a  wall  function  formulation  to 
reduce  the  need  for  grid  resolving  the  flow  in 
the  sublayer  portion  of  a  turbulent  boundary 
layer.  With  this  approach,  the  inner  region  of  the 
boundary  layer  is  modeled  by  an  analytical 
function  that  is  matched  with  the  numerical 
solution  in  the  outer  region.  The  wall  function 
formulation  has  the  advantage  of  1)  significantly 
reducing  the  memory  requirement  by 
eliminating  a  large  portion  of  cells  normally 
required  to  resolve  the  sub-layer,  and  2) 
improving  overall  convergence  by  removing  the 
thinner,  more  highly  stretched  cells  which  add 
stiffness  to  the  solution  process.  More  details 
are  presented  in  Ref.  [12], 

Wake  flow  -VGRIDns  does  not  presently 
support  the  generation  of  thin-layered 
tetrahedral  field  cells  for  resolving  the  wake 
flow  behind  a  wing  trailing  edge  (TE).  Thus,  it 
is  difficult  to  enforce  adequate  grid  resolution 
downstream  of  the  TE  without  resorting  to 
excessive  numbers  of  cells.  A  special  B.C.  [17] 
has  been  developed  to  mimic  the  relieving  effect 
of  a  blunt-base  wake  on  a  coarse  grid,  which  is 
useful  for  cases  where  the  wing  has  a  thick  TE. 
The  approach  is  to  introduce  a  solution-defined 
transpiration  velocity  on  the  blunt-base 
boundary  faces  to  provide  a  smooth  departure  of 
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the  flow  past  the  comer.  This  B.C.  has  been 
tested  and  is  used  extensively  with  inviscid 
flows  and  wall  function  applications. 

Jet-engine  -  USM3Dns  supports  a  B.C.  for 
modeling  propulsion-induced  effects  of  up  to 
four  turbofan,  turbojet,  or  jet-nozzle 
components.  The  model  [18]  computes  inflow 
and/or  outflow  B.C.’s  from  user  prescribed 
values  for  the  total  pressure  and  temperature 
change  across  or  within  the  combustor,  the  exit 
pressure,  the  fuel  fraction,  and  the  direction 
cosines  for  the  jet  exit.  The  inlet  mass  flow  is 
automatically  balanced  with  that  of  the  nozzle. 

Propeller  -  A  propeller/rotor  B.C.  has  been 
constructed  based  on  an  actuator  disk  model 
[19].  Since  VGRIDns  does  not  presently  support 
a  capability  to  generate  volume  grids  around 
‘zero-thickness’  surfaces  by  itself,  a  novel 
procedure  has  been  developed  for  this  purpose. 
The  procedure  provides  necessary  pre-  and  post¬ 
processing  utilities  for  VGRIDns  to  generate 
volume  grids  around  a  zero-thickness  rotor.  The 
rotor  is  composed  of  two  distinct  discretized 
surfaces,  designated  as  rotor  front  (pointing 
towards  the  incoming  flow  to  the  rotor)  and 
rotor  back  (pointing  in  opposite  direction  to 
rotor  front)  respectively,  which  are  collapsed 
onto  each  other.  In  USM3Dns,  a  subsonic  exit 
flow  boundary  condition  has  been  imposed  on 
the  rotor  front  surface  whereas  a  subsonic  inlet 
boundary  condition  has  been  imposed  on  the 
rotor  back  while  enforcing  mass  conservation. 
These  boundary  conditions  are  computed  based 
on  the  rotor  properties  provided  by  a  user  input. 
The  properties  include  thrust,  torque,  direction 
of  rotation,  size  and  location  of  each  of  the 
rotors/propellers.  Provision  has  been  made  so 
that  a  user  can  simulate  a  uniform  or  radially 
varying  pressure  and/or  swirl.  Up  to  four 
independent  rotors  can  be  simulated  in  a  single 
computational  domain. 

2.2.3  Parallel  Processing. 

The  USM3Dns  flow  solver  has  been  parallelized 
for  efficient  operation  on  both  shared  memory 
systems  and  a  cluster  of  UNIX  workstations 
connected  by  a  local  area  network  [20].  The 


tetrahedral  grid  must  be  partitioned  into  a  user 
specified  number  of  contiguous  zones  with  an 
approximately  equal  number  of  cells.  Grid 
partitioning  is  accomplished  by  a 
mathematically  rigorous  "spectral  bifurcation" 
graph  partitioning  technique  called  Metis  [21] 
that  has  been  adapted  to  the  TetrUSS  system. 

Communication  between  partitions  is 
accomplished  through  Message  Passing 
Interface  (MPI)  calls  embedded  into  the  solver. 
Efficiency  is  achieved  through  an  innovative  use 
of  specialized  data  structures,  send  and  receive 
arrays,  non-blocking  communication,  and 
"cache-friendly"  local  renumbering  of  cells  in  a 
partition. 

The  efficiency  of  the  parallelized  solver  is 
demonstrated  on  the  Numerical  Aerodynamic 
Simulator  (NAS)  Origin  2000  computer  using  a 
generic  business-jet  configuration  in  Fig.  2.  The 
geometry  is  comprised  of  a  fuselage,  a  wing, 
and  an  engine  nacelle  mounted  on  the  rear  of  the 
fuselage.  The  global  grid  contains  1,610,507 
tetrahedral  cells,  and  resolves  the  boundary  layer 
with  a  y+=50  using  the  wall  function.  The  flow 
conditions  are  Moo=0.85  and  Recref=3  million. 
Each  case  was  run  for  1500  cycles. 

A  super-linear  increase  in  speed  relative  to 
the  ideal  scaled  level  is  observed  in  Fig.  2  for  up 
to  200  processors.  The  speed-up  efficiencies  are 
referenced  to  a  10-processor  solution,  which 
was  the  minimum  number  of  processors  on 


Fig.  2  Efficiency  of  USM3Dns  parallel  flow  solver  on 
Origin  2000  for  a  generic  business-jet  configuration. 
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which  a  partitioned  grid  of  this  size  would  fit 
into  processor  memories.  It  is  assumed  that  a 
10-processor  solution  gives  a  ten-fold  increase 
in  speed  as  compared  to  a  single-processor 
solution.  The  super-linear  performance  is 
attributed  to  an  increased  cache  efficiency 
resulting  from  an  increasingly  larger  fraction  of 
cells  of  any  given  partition  fitting  into  cache 
memory  of  a  processor.  The  reduced  efficiency 
at  200  processors  illustrates  the  potential  of 
under  filling  the  cache  with  smaller  partitions 
such  that  the  overhead  from  inter-zonal 
communication  overrides  the  earlier  benefits 
realized  from  cache  efficiency. 

It  is  important  to  note  that  the  solution 
histories  were  essentially  identical  for  any 
number  of  processors.  Thus,  the  speed  benefits 
realized  in  Fig.  2  come  with  no  penalty  in 
accuracy  or  global  convergence. 

2.2.4  Low-Mach  Preconditioning 

The  flow  analysis  capability  of  USM3D 
has  been  extended  for  the  low-speed  flow 
regime  by  implementing  a  local  preconditioning 
approach  [22],  A  preconditioner  matrix  of 
Weiss-Smith  [23]  has  been  used  to  pre-multiply 
the  time  derivatives  of  the  governing  equations 
such  that  a  well-conditioned  system  of 
eigenvalues  is  rendered  at  low  speeds. 
Applications  with  the  code  have  demonstrated 
convergence  behavior  and  solution  quality  that 
is  nearly  independent  of  the  Mach  number.  The 
low-Mach  preconditioning  capability  is 
currently  available  only  in  the  serial  version  of 
USM3Dns. 

2.3  Modular  Engineering  Capabilities 

TetrUSS  includes  a  modular  capability  [24]  for 
computing  aeroelastic  effects  [25],  iterative 
design  [26],  and/or  interactive  boundary  layer 
[27],  Each  component,  illustrated  in  Fig.  3,  is 
maintained  independently  and  is  coupled  to  the 
system  by  pre-  and  post-processing  utilities.  The 
capabilities  can  all  be  used  simultaneously,  or  in 
any  combination. 

During  the  script-driven  update  cycle,  a 
solution  file  is  written  by  the  flow  solver  and  is 


used  along  with  the  grid  file  by  the  sub- 
modules.  Both  the  aeroelastic  and  design 
features  involve  grid  movement,  whereas  the 
interactive  boundary  layer  (IBL)  feature 
generates  transpiration  boundary  condition 
velocities. 

The  aeroelastic  component  converts  the 
loads  from  the  flow  file  into  input  for  the 
structural  code.  The  present  options  for 
structural  modeling  include  a  coupling  with  the 
ELAPS  code  [28]  and  the  ENSAERO  system 
[29].  The  ELAPS  code  is  a  computationally 
efficient  method  based  on  plate  model  to  solve 
for  a  polynomial  global  displacement  function 
using  the  Rayleigh-Ritz  method.  Higher-level 
Finite  Element  Model  (FEM)  structural 
representations  can  be  input  through  ENSAERO 
via  modal  or  stiffness  matrices. 

The  iterative  design  capability  is  provided 
by  the  Constrained  Direct  Iterative  Surface 
Curvature  (CDISC)  method  [26],  CDISC  uses  a 
knowledge-based  approach  to  modify  target 
pressure  distributions  that  are  used  by  the 
underlying  DISC  method  to  match  flow 
constraints.  The  basic  DISC  design  method  is  an 
iterative  approach  to  modifying  an  aerodynamic 
surface  to  obtain  a  given  surface  pressure 
distribution.  It  is  based  on  analytically  derived 
expressions  relating  changes  in  surface 
curvature  to  a  change  in  surface  pressure 
coefficient,  or  more  recently  surface  Mach 
number.  Since  this  relationship  is  known 
apriori,  the  design  can  be  converged  very 
efficiently  in  parallel  with  the  flow  solution. 


&  Can  substitute  other  flow  solvers 


Fig.  3  Schematic  of  modular  TetrUSS  system 
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Two  2D  boundary  layer  modules  are 
currently  available.  The  first  is  a  proprietary 
code  based  on  the  inverse  boundary  layer 
methodology  of  Cebeci,  et.  al.  [30].  In  this 
method,  the  boundary  layer  is  solved  iteratively 
as  part  of  the  2D  boundary  layer  solution.  Using 
the  boundary  layer  edge  values,  the  surface 
transpiration  velocities  are  calculated,  which  in 
turn,  are  used  as  a  boundary  condition  by  the 
inviscid  flow  solver,  thus  indirectly  accounting 
for  the  boundary  layer  displacement  thickness. 

The  second  module  is  in  the  public 
domain,  and  uses  the  Green’s  Lag  Entrainment 
method  [31].  This  method  employs  the 
momentum  integral  equation,  the  entrainment 
equation  and  an  equation  for  the  streamwise  rate 
of  change  of  entrainment  coefficient  to  calculate 
turbulent  boundary  layer  characteristics. 

Examples  of  the  IBL  and  static  aeroelastic 
capabilities  will  be  shown  in  section  3.2. 

2.4  Graphic  Postprocessing 

VIGPLOT  [32]  is  an  interactive,  menu-driven 
post-processing  program  for  extraction  and 
display  of  data  on  unstructured  (tetrahedral) 
grids.  It  displays  grids  and  flow  quantities  on 
either  boundary  or  user  defined  arbitrary  planes 
in  the  field.  Flow  quantities  can  be  displayed 
using  line  or  filled  contours,  velocity  vectors 
and  particle  traces.  Some  unique  features 
include  dynamic  memory  allocation,  a  ‘probe’ 
option  that  lets  a  user  query  the  local  value  of  a 
certain  displayed  fluid  dynamic  quantity,  ability 
to  plot  two  different  sets  side-by-side  and  a 
journaling  (log  file)  capability.  Grid  post¬ 
processing  options  allow  a  user  to  isolate  and 
display  ‘bad’  grid  cells  and  to  visually  confirm 
boundary  conditions  before  proceeding  with  the 
flow  solution.  Several  hardcopy  output  options 
are  available. 

Other  graphic  postprocessors  can  also  be 
used  with  TetrUSS.  Output  files  are  supported 
for  the  well-known  FAST  and  Tecplot  codes. 


2.5  Utility  Codes 

Several  useful  utility  codes  are  maintained 
through  a  common  interface  called  “usgutil”. 
They  are: 

1.  USM3Dns  to  Tecplot  converter, 

2.  Mirroring  of  grid  or  VGRIDns  input  file, 

3.  Extracting  cross-sectional  or  field  data 
for  density,  velocity,  energy,  pressure, 
pressure  coefficient  (Cp),  Mach  number, 
temperature,  entropy,  or  total  pressure, 
and, 

4.  Providing  an  estimate  for  grid  spacing 
and  stretching  parameters  used  in 
VGRIDns  for  viscous  grids. 

3  Selected  Applications 

An  assessment  of  the  TetrUSS  technology  is 
made  using  selected  examples  that  emphasize 
various  features.  Two  2D  flat-plate  cases  are 
presented  to  address  solution  accuracy  for 
laminar  and  turbulent  flows.  Five  3D 
simulations  are  shown  to  assess  1)  Euler/IBL 
coupled  with  aeroelasticity  on  a  generic 
transonic  transport  wing/body  configuration,  2) 
N-S  subsonic  laminar  vortical  flow,  3)  N-S 
transonic  separated  flow  on  an  unconventional 
aircraft,  4)  Euler  and  N-S  comparison  on  a 
complex  fighter  configuration,  and  5)  N-S 
supersonic  flow  with  coupled  aeroelasticity  on 
supersonic  transport  configuration.  The  Spalart- 
Allmaras  turbulence  model  is  used  in 
conjunction  with  the  wall  function  for  all 
turbulent  N-S  computations. 

3.1  2D  Test  Cases 

3.1.1  Laminar  Flat-Plate  Boundary  Layer 

A  laminar  Navier-Stokes  USM3Dns  solution  is 
computed  on  a  flat  plate  for  comparison  with  the 
exact  Blasius  boundary  layer  solution  to  validate 
the  numerical  treatment  of  the  viscous  stencil.  A 
quasi-2D  “channel”  grid  was  constructed  by 
subdividing  each  hexahedral  cell  of  a  49x41x3 
structured  grid  into  six  tetrahedral  cells.  The 
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resulting  unstructured  grid  has  23040 
tetrahedral  cells. 

The  flow  solution  was  computed  at 
Moo=0.5  and  a  ReL=10000.  A  comparison  of  the 
velocity  profiles  is  shown  in  Fig.  4  for  several 
chord  stations.  The  transformed  computed 
profiles  collapse  well  between  longitudinal 
stations  0.20L  and  0.95L,  and  are  in  good 
agreement  with  the  exact  Blasius  solution. 

3.1.2  Turbulent  Flat-Plate  Boundary  Layer 

A  flat  plate  turbulent  boundary-layer  solution  at 
Moo=0.5  and  ReL=2  million  is  used  to  assess  the 
accuracy  of  the  wall  function  in  predicting 
turbulent  skin  friction.  A  second  quasi-2D 
“channel”  grid,  similar  to  that  for  the  laminar 
case,  was  constructed  from  a  49x19x3 
hexahedral  grid.  The  final  unstructured  grid 
contains  10368  tetrahedral  cells.  The  initial 
normal  spacing  yields  a  y+=80  at  x/L=0.5. 

Fig.  5  reveals  very  good  agreement 
between  the  computed  and  theoretical  skin 
friction  coefficient,  Cf.  More  details  are 
presented  for  similar  comparisons  in  Ref.  [12]. 

3.2  3D  Test  Cases 

3.2.1  NTF  Pathfinder  I  -  Euler  with  IBL  and 
aeroelasticity 

The  transonic  aerodynamics  of  transport 
configurations  with  supercritical  airfoils  can 
present  a  formidable  challenge  to  predict 
computationally.  Not  only  are  the  viscous 
effects  important,  but  also  the  load-induced 
deformation  of  the  thin  high-aspect  ratio  wings 
experienced  during  wind-tunnel  testing  and 
flight.  The  location  of  the  transonic  shock  is 
extremely  sensitive  to  both  factors.  Thus, 
viscous  and  static  aeroelastic  effects  must  be 
accounted  for  in  order  to  achieve  a  meaningful 
correlation  for  this  class  of  vehicle. 

The  modular  TetrUSS  system  has  been 
applied  to  the  National  Transonic  Facility  (NTF) 
Pathfinder  I  wing/body  configuration  at  a 
transonic,  high  Reynolds  number  flow  condition 
[25],  Viscous  effects  are  estimated  from  a 
coupled  Euler/IBL  solution,  and  static 
aeroelastic  effects  from  a  simple  plate 
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Fig.  4  Comparison  of  exact  Blasius  solution  for 
laminar  flow  over  flat  plate  with  USM3Dns  result. 
Moo=0.5,  ReL=10000. 
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Fig.  5  Comparison  of  computed  vs.  theoretical  skin 
friction  coefficient  for  turbulent  flat-plate  flow. 
M^O.5,  ReL=2  million. 

representation  constructed  for  the  ELAPS  code. 
An  additional  single-point  calibration  has  been 
made  to  the  ELAPS  model  using  static  bending 
data  measured  on  the  wind-tunnel  model.  The 
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Fig.  6  Effect  of  aeroelastic  deformation  on  Cp 
correlation  for  NTF  Pathfinder  I  wing/body 
configuration  at  a  transonic,  high  Reynolds  number 
flow  condition 

strength  of  this  approach  is  that,  with  a  single¬ 
point  pretest  calibration  of  a  simple  structural 
representation,  the  static  aeroelastic  effects  can 
be  accounted  for  in  CFD  calculations  across  a 
range  of  test  conditions.  No  prior  knowledge  of 
the  model  deformation  during  the  wind-on  test 
is  required. 

Fig.  6  depicts  a  comparison  of  the  rigid 
and  deformed  Pathfinder  solutions  with 
experimental  pressure  data  at  two  span  stations. 
The  deformed  solution  is  converged  from  the 
rigid  geometry  after  25  iterations  through  the 
modular  system.  Ten  flow  cycles  were 
performed  between  each  grid  deformation.  This 
result  confirms  that  the  accounting  for  static 
aeroelastic  effects  is  critical  to  a  good 


Fig.  7  Comparison  of  computed  and  experimental 
surface  pressure  data  and  surface  triangulation  and 
"oil-flow"  patterns.  \U=0.3.  a=20.5  deg.,  and 
Recref=900,000. 

correlation  with  the  data  for  this  class  of 
vehicle. 

3.2.2  Hummel  Delta  Wing  -  N-S,  Subsonic, 
Laminar  Vortical  Flow 

A  laminar  Navier-Stokes  flow  solution  was 
computed  on  the  aspect-ratio  1  delta  wing  tested 
by  Hummel  [33].  Flow  conditions  were 
Moo=0.3,  oc=  20.5  deg.,  and  Recref=900,000.  A 
grid  of  730,454  tetrahedra  was  generated  with 
normal  spacing  distributed  near  the  surface  to 
yield  approximately  10  nodes  (30  tetrahedra)  in 
the  mid-chord  boundary  layer  for  laminar  flow. 
Anisotropic  stretching,  evident  in  the  surface 
grid  shown  in  Fig.  7,  is  utilized  to  reduce  the 
chordwise  density  cells  in  the  grid.  This  resulted 
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in  a  7-fold  reduction  in  grid  size  compared  to 
standard  isotropic  triangles.  A  companion 
structured-grid  computation  was  performed  with 
the  well-established  CFL3D  code  [34]  on  a 
65X65X33  (span-radial-chordwise)  H-O  grid 
for  comparison. 

Fig.  7  portrays  the  surface  "oil-flow" 
pattern  and  a  comparison  of  the  spanwise 
distribution  of  pressure  coefficient,  Cp,  at  four 
chord  stations.  The  coalescing  and  diverging 
streamlines  in  the  computed  flow  patterns  show 
evidence  of  the  primary,  secondary,  and  tertiary 
vortices.  Good  agreement  is  noted  between  the 
unstructured  and  structured  solutions,  and  with 
the  experimental  data  of  Hummel. 

3.2.3  Joined  Wing  -  N-S,  Transonic, 

Separated  Flow 

A  computation  was  made  on  the  Joined- Wing 
configuration  published  in  Ref.  [35]  in  order  to 
calibrate  USM3Dns  on  an  unconventional 
aircraft  configuration  operating  at  transonic, 
separated  flow  conditions.  A  thin-layered 
tetrahedral  grid  of  1,400,984  cells  (246,891 
nodes)  was  generated  on  the  half-configuration. 
Grid  size  is  kept  small  by  utilizing  anisotropic 
cell  stretching  which  results  in  a  factor-of-three 
fewer  cells  than  an  isotropic  grid  with  similar 
chordwise  resolution.  The  normal  grid  spacing 
was  sized  for  wall  function  to  yield  a  nominal 
midchord  y+=50  for  the  first  node  above  the 


Fig.  8  Surface  triangulation  and  "oilflow"  patterns 
for  a  joined-wing  configuration,  magenta  =  low 
pressure  and  green  =  high  pressure. 


surface  and  18  to  20  tetrahedra  across  the 
boundary  layer.  The  grid  was  completed  in  three 
days. 

Fig.  8  depicts  the  surface  triangulation  and 
representative  "oilflow"  patterns  on  the  joined- 
wing  configuration.  Significant  shock-induced 
and  trailing-edge  flow  separations  are  observed 
on  the  fore  and  aft  wings.  Severe  flow  reversal 
is  evident  on  the  tip  panel.  The  surface  flow 
patterns  are  in  good  agreement  with 
experimental  oilflow  data  (not  shown)  obtained 
in  the  NASA  Langley  Research  Center  16-Foot 
Transonic  Tunnel. 

Fig.  9  portrays  the  longitudinal  pressure 
coefficient  (Cp)  distribution  along  three 
representative  spanwise  stations  of  the  fore  and 
aft  wings,  and  tip  panel.  The  computed 
pressures  are  in  good  agreement  in  each  flow 
region  with  the  experimental  data.  The 
computation  required  250MW  of  memory  and 
27  CPU  hours  on  a  CRAY  C90.  With 
multitasking  on  10  processors,  the  solution  was 
completed  in  5  wall-clock  hours. 

3.2.4  F-16  Aircraft  with  Stores  -  N-S  and 

Euler 

Unstructured  transonic  Navier-Stokes 
computations  are  presented  from  Ref.  [36]  for 
the  complete  F-16  aircraft  configuration  in  an 
ongoing  assessment  of  TetrUSS  for  aerodynamic 
analysis  of  very  complex  geometries  and  flow 


Fig.  9  Comparison  of  Cp  distributions  for  joined-wing 
configuration  on  forward  and  aft  wings,  and  tip  panel 
at  transonic  Mach  number  and  wind-tunnel  Reynolds 
number. 
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Fig.  10  Viscous  tetrahedral  grid  on  complete  F-16 
aircraft  with  external  stores,  1,428,779  cells.  Note 
thin-layer  clustering  on  symmetry  plane. 

fields.  A  thin-layered  tetrahedral  grid  of 
1,428,779  cells  (255,959  nodes)  was  generated 
for  a  full  F-16  aircraft  configuration  with 
VGRIDns  (see  Fig.  10).  A  companion  "in viscid" 
grid  of  1,111,762  cells  (not  shown)  was  also 
generated  for  comparison  purposes.  Grid  sizes 
were  kept  small  by  utilizing  anisotropic  cell 
stretching.  The  initial  viscous  grid  was 
completed  within  approximately  40  to  60  hours 
of  labor. 

Turbulent  flow  solutions  were  computed 
with  the  USM3Dns  flow  solver  at  angles  of 
attack  (a)  of  0,  2,  4,  and  8  degrees,  Moo=0.95, 
and  Recref=2  million  based  on  mean 
aerodynamic  chord.  The  normal  grid  spacing 
was  sized  for  the  wall  function  to  yield  a 
nominal  midchord  y+=30  for  the  first  node 
above  the  surface  and  18  to  20  tetrahedra  across 
the  boundary  layer. 

Fig.  1 1  compares  the  longitudinal 
distribution  of  Cp  from  Euler,  N-S,  and 
experimental  data  along  the  inboard  and 
outboard  sides  of  the  outer  ‘finned’  store.  As 
expected,  the  inviscid  shock  is  too  strong  and 
too  far  aft  compared  to  data,  whereas  the  N-S 
solution  results  in  a  softened  expansion  and 
brings  the  aft-shock  into  better  agreement  with 
data.  The  quality  of  correlation  for  the  N-S 
solution  is  marginal  due  to  insufficient  grid 
resolution  around  the  store  fins. 


A  typical  N-S  solution  required  254MW  of 
memory  and  1 1  hours  of  CRAY  C90  time.  With 
a  multitasking  efficiency  of  6  out  of  10 
processors,  each  case  was  completed  in  2 
wallclock  hours. 

3.2.5  HSR  Cruise-  N-S,  Supersonic, 
Aeroelastic 

Ref.  [37]  presents  a  simple  technique  that 
accounts  for  aeroelastic  deformations 
experienced  by  High-Speed  Research  (HSR) 
wind-tunnel  models  within  CFD  computations. 
During  wind-tunnel  tests  of  the  low  aspect-ratio 
planforms  characteristic  of  HSR  configurations 
(Fig.  12),  the  load  induced  displacement  and 
twist  of  the  thin  outer  wing  panels  tends  to 


x/^store 


Fig.  11  Longitudinal  distribution  of  surface  pressure 
coefficient  on  F-16  generic  finned-store  body  at  Mach 
0.95  and  cx=4  degrees.  Store  nose  at  x/Lstore=0,  and 
x/Lst0re=l  slightly  aft  of  store  fins. 


Fig.  12  Viscous  tetrahedral  grid  on  complete  HSR 
configuration,  1,428,427  cells.  Note  thin-layer 
clustering  on  symmetry  plane 


241.11 


N.  Frink,  S.  Pirzadeh,  P.  Parikh,  M.  Pandya  and  M.  Bhat 


unload  those  panels.  This  results  in  a  decrease  in 
lift  and  increase  in  pitching  moment.  The 
variation  of  drag  with  lift  is  not  adversely 
affected.  Thus,  it  is  important  to  include  the 
model  deformation  within  numerical 
simulations  to  remove  the  noted  aeroelastic- 
induced  inconsistencies. 

The  USM3Dns  N-S  grid  and  solution  is 
coupled  with  the  ELAPS  code  within  the 
modular  TetrUSS  framework.  The  ELAPS 
model  is  calibrated  against  the  static  bending 
characteristics  of  the  wind-tunnel  model  in  a 
manner  similar  to  that  described  in  Sec.  3.2.1. 
At  the  time  of  this  work,  the  viscous  grid 
movement  described  in  Sec.  2.1.3  had  not  been 
automated  and  was  accomplished  manually 
through  POSTGRID. 

Fig.  13  depicts  the  CFD  correlations  with 
experimental  force  and  moment  data  at  M„=2.4 
and  Recref=6.4  million.  Note  that  the  USM3Dns 
result  for  the  ‘rigid’  geometry  over-predicts  lift 
and  significantly  under-predicts  pitching 
moment.  The  drag  coefficient  is  in  good 
agreement  and  includes  corrections  for  nacelle 
base  pressure  and  for  trip  drag. 

The  ELAPS  calibration  point  is  denoted  by 
the  solid  triangle  on  the  pitching-moment  curve. 
The  remaining  open  triangles  were  computed 
with  USM3Dns  using  the  calibrated  ELAPS 
model.  Note  the  good  agreement  of  both  the  lift 
curve  and  the  pitching  moment  across  the  range 
of  experimental  data.  Note  further  the  excellent 
agreement  with  the  CFL3D  computation  [38]. 
The  CFL3D  result  was  computed  on  a  deformed 
structured  grid  that  matched  the  optically 
measured  shape  of  the  wind  tunnel  model. 

4  Status  of  Emerging  Capability 

The  following  contains  a  brief  synopsis  of 
published  work  that  is  currently  underway. 
These  capabilities  are  not  presently  in  the 
TetrUSS  system. 

4.1  Solution  Adaptive  Grids 

Solution  adaptive  grid  technology  is  a  powerful 
tool  in  CFD  that  provides  three  main  benefits 
for  solving  complex  aerodynamic  problems: 


O  Experiment 
— Q—  USM3Dns:  Rigid 
-A-  USM3Dns:  ELAPS 
A  Calibration  Point 
— O —  CFL3D  (Deformed) 


a, deg  CD  CM 

Fig.  13  Effect  of  calibrated  USM3Dns/ELAPS  on 

force  and  moment  correlation  for  HSR  configuration. 

Mach=2.4,  Rec=6.4  million. 

increased  automation,  better  solution  accuracy, 
and  computational  efficiency.  An  adaptive 
unstructured  grid  refinement  technique  has  been 
developed  and  applied  successfully  to  three- 
dimensional  inviscid  flow  test  cases  [39].  The 
method  is  based  on  a  combination  of  grid 
subdivision  and  local  remeshing.  With  this 
approach,  the  surface  triangulation  is  first  "h- 
refined"  locally,  and  the  tetrahedral  grid  in  the 
field  is  then  partially  remeshed  at  locations  of 
dominant  flow  features.  The  grid  movement 
technique,  described  in  an  earlier  section  of  this 
paper,  is  also  incorporated  in  the  present 
adaptive  grid  package  for  geometric  adaptation 
of  volume  grids  to  moving  or  deforming 
surfaces. 

4.2  Advanced  Turbulence  Models 

Two  advanced  turbulence  models  have  been 
implemented  into  a  working  version  of 
USM3Dns  [40]  and  are  currently  being  tested. 
The  models  are  based  on  modified  versions  of 
the  original  Jones  and  Launder  k-e  two-equation 
turbulence  model  [41]  and  the  Girimaji 
algebraic  Reynolds  stress  model  [42].  The 
current  one-equation  S-A  model  has 
shortcomings  in  predicting  large-scale  separated 
flows  and  shear  flow.  This  work  is  motivated  by 
the  need  for  better  predictions  of  those 
phenomena  and  to  establish  the  code  framework 
for  investigating  other  two-equation  models. 
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5  Concluding  Remarks 

TetrUSS  is  a  synthesis  of  key  state-of-the-art 
unstructured- grid  technologies  into  a  loosely 
coupled  software  system  to  form  a  user-friendly 
aerodynamic  analysis  and  design  tool.  Salient 
features  and  capabilities  include: 

•  The  interactive  GridTool  code  provides  a 
key  bridge  between  CAD  surface  definitions 
and  the  VGRIDns  grid  generator. 

•  The  VGRIDns  code  is  an  established 
computer  program  for  automatic  generation 
of  tetrahedral  unstructured  grids.  It  produces 
thin-layered  tetrahedra  suitable  for 
computing  Navier-Stokes  flow  solutions, 
and  high-quality  anisotropically  stretched 
tetrahedral  cells  that  dramatically  reduce  the 
overall  grid  size  without  compromising 
solution  accuracy.  A  robust  ‘viscous’  grid 
movement  capability  is  also  available. 

•  USM3Dns  produces  efficient  and  accurate 
Euler  and  Navier-Stokes  flow  solutions  on 
vector  processors  and  massively  parallel 
computers.  The  Spalart-Allmaras  one- 
equation  turbulence  model  is  presently 
utilized  and  is  routinely  coupled  with  a  wall 
function.  Special  boundary  conditions  are 
provided  for  wake  flow,  jet  engine 
simulation,  and  propellers.  A  low-Mach 
number  preconditioning  version  is  available. 

•  Modular  engineering  capabilities  are 
provided  for  low-cost  Euler/IBL  viscous 
analysis,  iterative  design  using  the  CDISC 
code,  and  static  aeroelasticity. 

•  Several  supportive  utility  codes  are  available 
under  a  single  interface. 

In  summary,  TetrUSS  has  enabled  rapid 
solutions  to  many  complex  aerodynamic 
problems.  Applied  aerodynamicists  from  a 
broad  cross-section  of  background  and  skill 
level  have  successfully  applied  the  system  to 
their  tasks. 

Work  continues  toward  upgrading  TetrUSS 
capabilities.  The  most  near-term  additions  will 
be  solution-adaptive  grids,  advanced  two- 
equation  turbulence  models,  and  porting  to  PC 
clusters. 
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